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Abstract. We study ergodic quadratic optimal stochastic control problems for an affine state 
equation with state and control dependent noise and with stochastic coefficients. We assume 
stationarity of the coefficients and a finite cost condition. We first treat the stationary case and 
we show that the optimal cost corresponding to this ergodic control problem coincides with the 
one corresponding to a suitable stationary control problem and wo provide a full characterization 
of the ergodic optimal cost and control. 
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1. Introduction 

In this paper we study an ergodic quadratic control problem for a linear affine equation with both 
state and control dependent noise, and the coefficients of the state equation, allowed to be random, 
are assumed to be stationary. We continue our previous work [4], where the infinite horizon case 
and the ergodic case are studied but no characterization of the ergodic limit was given. The main 
result of the present paper is to obtain the characterization of the ergodic limit, see Theorem 13.51 
when the coefficients are stationary in a suitable sense, see [TTl and section 2 below. 

The main tool will be Backward Stochastic Riccati Equations (BSREs): such equations are 
naturally linked with stochastic optimal control problems with stochastic coefficients. The first 
existence and uniqueness result for such a kind of equations has been given by Bismut in [2], but 
then several works, see e. g. [3], [B], [7], [5], [5] followed. Only very recently Tang in [TD] solved the 
general non singular case corresponding to the linear quadratic problem with random coefficients 
and control dependent noise. In [3], we have studied the infinite horizon case and the ergodic case 
namely, we have considered a cost functional depending only on the asymptotic behaviour of the 
state (ergodic control). 

Starting from this point, in this paper we first consider the stationary problem: minimize over all 
admissible controls the cost functional 

j\u,X) = E / [| V^TX.P + 
Jo 

The control u is stationary and X is the corresponding solution of the state equation 

d d 

dXt = AtXtdt + Btutdt + ^ ClXtdWl + ^ D\utdWl + ftdt. (1.1) 

i=l i=l 

We denote the optimal cost for the stationary problem by . 

The main technical point of this paper is to prove that the closed loop equation for the stationary 
control problem, admits a unique stationary solution, see proposition 12.101 

In order to study the ergodic control problem, we first consider the discounted cost functional 

r+oo 

r{0,x,u)=E e-^"'[{S,X°'^'^,X°'-'^) + \us\'']ds, (1.2) 
Jo 



where X is solution to equation 

d 

dXs^{A,Xs + B,Us)ds + Y,{ClXs + Dlus)dWl + fsds, s>0 3^ 

i—l 

Xq = X. 

A, B, C and D are bounded random and stationary processes and / e L^{fl x [0;+oo),R"), 
moreover we assume suitable finite cost conditions. It is proved in [4J that in general, without 
stationarity assumptions, 

r + co 



p+oc d d 

•^0 t=i *=i 



Starting from this point, we show here that in the stationary case 

lim p, .Q2a J (x) — 

Then we consider the "true" ergodic optimal cost, we minimize the following functional 

J(a;, u) ~ lim p_.p2Q; Jfx. u) 
over all u e Z^, see p.Sp for the definition of U. We are able to prove that 

inf_ J(x,it) = J^{u). 

and to the characterize the optimal ergodic control, see lemma WM and theorem 13.51 

2. Linear Quadratic optimal control in the stationary case 

Let (f2,JF, P) be a probability space and assume that W : (—00, +00) — > M is a d-dimensional 
brownian motion defined on the whole real axis. Let \ J-t\tf^(-oo,^-oa) its natural filtration completed. 
For all s, t e K with t > s we denote by Ql the ti-field generated by {WV — M^s, 5 < r < t}. Notice 
that for all s S M, \Qt\t>s ^ filtration in (ri,JF). Finally we assume that for all s < 0, ^/q C T^- 
Next we set a stationary framework: we introduce the semigroup (Qt)tf^v. of measurable mappings 
dt : ^ verifying 

(1) 00 = Id, 9toes = 9t+s, for all t,seR 

(2) Ot is measurable: {fi,f't) ^ (f^,^o) and {{Ot e A} : A e Tq} ^ Tt 

(3) ¥{9t £A} = ¥{A) for aU AeTo 

(4) Wt o 0, = Wt+s - Ws 

According to this framework we introduce the definition of stationary stochastic process. 

Definition 2.1. We say that a stochastic process X : [0,oo[x^ ^ W"\ is stationary if for all s G M 

Xt o 0s = Xt+s P-a.s. for a.e. t > 

We assume all the coefficients A, B, C, D and 5 to be stationary stochastic processes. Namely 
on the coefficients we make the following assumptions: 

Hypothesis 2.2. 

Al) A : [0,+cx)) X n ^ K"^", B : [0,+oo) x -> ]R"^'=, C : [0,+oo) xfl^ M"^", i = l,...,d 
and : [0, +c») x O ^ R"^*^, i = l,...,d, are uniformly bounded process adapted to the 
filtration {^t}t>o- 

A2) 5* : [0, +00) x — !■ M"^" is uniformly bounded and adapted to the filtration {J-'t}^yQ and it 
is almost surely and almost everywhere symmetric and nonnegative. Moreover we assume 
that there exists P > such that S > (31 . 

A3) A, B, C , D and S are stationary processes. 

In this case we immediately get: 



2 



Lemma 2.3. Fix T > and let hypothesis \ 2.2\ holds true. Let {P, Q) be the solution of the finite 
horizon BSRE 



-dPt^GiAt,Bt,Ct,DuSt;PuQt)dt + J2QldWi, te [0,T] 
Pt = Pt- 



i=l 



(2.1) 



For fixed s > we define P{t + s) = P{t)9s, Q{t + s) = Q{t)9s then {P,Q) is the unique solution in 
[s, T + s] of the equation 



a 

-dPt ^ G (^At,Bt,Ct,Dt;St;Pt,Qt) dt + J2QldWl, te[s,T^ 



(2.2) 



Pt — Pt ° ' 



In the stationary assumptions the backward stochastic Riccati equation 



dPt 



a 

AlPt + PtAt + St + J2 {{CiyPtCl + {ClY Qt + QtCi) 



i=l 



dt + j2Qidwi 



i+j2{D^yp,Di 



-Ir 



PtBt + J2({GiyPtDl + QlDl 



(2.3) 



dt, 



(2.4) 



admits a minimal sohition (P, Q), in the sense that whenever another couple (P, Q) is a solution to 
the Riccati equation then P — P is a non- negative matrix, see also Corollary 3.3 in [5J and definiton 
3.2 in [1]. This minimal solution (P, Q) turns out to be stationary. 

Proposition 2.4. Assume hupothesis \2.2l then the minimal solution {P,Q) of the infinite horizon 
stochastic Riccati equation (j2.3p is stationary. 



Proof. For all p > we denote by PP the solution of equation (|2.ip in [0, p] with final condition 
PP{p) = 0. Denoting by [pj the integer part of p, we have, following Proposition 3.2 in [S] that for 



all N for aU t e [0, [N + s\], P^ 
that by lemma 



[N+s\ 



< P^^' < Pt 



[N+s\+l 



-a.s.. Thus we can conclude noticing 



pN+s _ p 



N 



Thus letting TV — > +oo we obtain that for all t > 0, and s > 0: 

p{Pt+s^Ptoes} = 1. 

Now Pt+s = Pt ° Os ^ Pt so if one consider (|2.ip in the intervall [s,T + s] with final data 
Pt+s and (|2.2p with final data Pt ° Og, by the uniqueness of the solution it follows that Qr = 
Qr, P — a.s. and for almost all r ^ [s,T + s]. □ 

We notice that in the BSRDE ()2.3p the final condition has been replaced by the stationarity 
condition on the solution process (P, Q). 

Next we give some definitions. 

Definition 2.5. We say that {A,B,C,D) is stabilizable relatively to the observations (or ^/S- 
stabilizable) if there exists a control u G ip([0, +oo) x il; M.^) such that for alH > and all x e M" 

/ + 00 
X*'"'") + \u,\^]ds < Mu,. (2.5) 

for some positive constant Mt^x where X*'^'" is the solution of the linear equation 

d 

dXs = {A,Xs + Bsu,)ds + {GIX, + D\u,) dWl s > 



(2.6) 



Xn = X. 
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This kind of stabilizability condition, also called finite cost condition, has been introduced in [S]. 
This condition has been proved to be equivalent to the existence of a minimal solution (P, Q) of the 
Riccati equation (|2.3p . Moreover whenever the first component P is uniformly bounded in time it 
follows that the constant Mt^x appearing in (|2.5|) can be chosen independent of time. 

Definition 2.6. Let P be a solution to equation (|2.3p . We say that P stabilizes {A, B,C, D) 
relatively to the identity I if for every t > and x G M" there exists a positive constant M, 
independent of t, such that 

/ + 00 
|X*'"^(r)pdr < M F-a.s., (2.7) 



where X*'^ is a mild solution to 
dXt 



E 



AXt-Btil + Y, {Dl)*PtDl PtBt + {Q\DI + {ClY PtDi) Xt 



ClXt^Dlll + Y{DiyPtDl\ lptBt + Y{QlDl + {CiyPtDM X 



1=1 

^ Xq^X 

From now on we assume that 



dt+ 



dWt. 



(2.8) 



Hypothesis 2.7. 

(i) {A, B, C, D) is \/S- stabilizahle; 

(ii) the process P is uniformly bounded in time; 

(iii) the minimal solution P stabilizes {A, B, C, D) with respect to the identity I . 

We refer to [1] for cases when P stabilizes {A, B, C, D) relatively to the identity /. Notice that, 
thanks to the stationarity assumptions the stabilizability condition can be simplified, see Remark 
5.7 of [5]. 

Next we study the dual (costate) equation in the stationary case. We denote by 
A{t,Pt,Qt) = - (^I + J2{DiyPtDl^ ^tBt+J2(QlDl + {CiyPtD\ 



Ht = At + BtA{t,Pt,Qt) , 
Kl^Cl + DlA{t,Pt,Qt). 
Thanks to Proposition 12.41 all the coefficients that appear in equation 



(2.9) 



(2.10) 



drt = -H;rtdt - Ptftdt - Y (Kl) * gldt + Y,9ldWl. t e [0, T] 

i=l i=l 

rr = 0. 

are stationary so exactly as before we deduce that for the solution {rT,gT) the following holds: 
Lemma 2.8. Let A, B, C, D and S satisfy hypothesis \2.'A and let f S (Jl x [0, +oo)) be a 



stationary process. Fix T > and rx G L^{fl,J^T', 



Let (r, g) a solution to equation 



drt = -H*rtdt - Ptftdt - J2 {Kl) * dldt + ^ffjdW^^ t e [0, T] ^^.n) 

i—l i—1 

{, rr = rx- 

For fixed s > we define rt+s = rt o 9s, gt+s = 9t ° then (r, g) is the unique solution in [s, T + s] 
of the equation 

d d 

d?t = -H^rtdt - Ptftdt - i^i) * 9ldt + Ydl'^^t, t e ['^, T + s] 



rr — tt ° ^ 
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Hence arguing as for the first component P, we get tliat tlie solution of tlic infinite horizon dual 
equation is stationary, as stated in the following proposition: 

Proposition 2.9. Assume hvvothesis \2.2\ and hypothesis \2.1\ then the solution {r'^,g^) of 

d d 

dn = -Hlndt - Ptftdt - {KlYgldt + Y.g\dWl, (2.13) 

1=1 i=l 

obtained as the pointwise limit of the solution to equation (|2.10|) is stationary. Moreover {r^,g^) & 
{Q X [0, 1] , M") X Ll,{nx [0, 1] , M"^'*) . 

Proof. The proof follows from an argument similar to the one in Proposition 4.5 in Stationarity 
of the solution {r\g''^) follows from the previous lemma. □ 
Again we notice that in the dual BSDE (|2.13p the final condition has been replaced by the 
stationarity condition on the solution process (r'' ,(?''). 

We need to show that in the stationary assumptions, the solution of the closed loop equation is 
stationary. By using notation (|2.9p , we consider the following stochastic differential equation, which 
will turn out to be the closed loop equation: 

d d d d 

dX, = HMs + Y^KlX^dWl + BsiBlrl + Y.DlgY)ds + f^ds + Y^DHBlr^ + Y,DlgY)dWl, 

(2.14) 

where {r^,g^) is the solution of the dual (costate) equation (|2.13p . 

Proposition 2.10. Assume huvothesis \2.2\ and hypothesis \2. 7| holds true then there exists a unique 
stationary solution of equation (|2.14p . 

d d 

Proof. We set /i = /, + 5.(5: + E^^^^") ^^^^ = ^^(^^^ + ^ = 1' 

1=1 1=1 
can extend for negative times letting for all t g [0, 1], flj^^t — fl ° 9^n, i ~ 1,2, N G 

We notice that |[_7v +oo) is predictable with respect to the filtration {Gi^^)t>-N- Therefore for 
all e N equation 

d 

dX-^ = H^X-^ds + + Eti KlX-^dWl + fids + Y^^f 'dWl, 
XZ^ = 0, 

admits a solution {X^^'^)t, defined for t > —N and predictable with respect to the filtration 
{g^'^)t>-N■ We extend X-^'° to the whole real axis by setting X~^'^ = for i < -N. We want 
to prove that, fixed t e M, {X^^'^)]s[ is a Cauchy sequence in L^(0). In order to do this we notice 
that for f > — iV + 1, X^ '° — X^ ^^''^ solves the following (linear) stochastic differential equation 



J-N+l J-N+1 

By the Datko theorem, see e.g. 0] and [S], there exist constants a,c> such that 

(E|A-^'° - A-^+i'°ni/2 < Ce-'^'^inXZ^-^.iy/^. 
So, fixed t e K and M, N eN, M > N sufliciently large such that -N < t, 



)dW: 



A/-1 A/-1 



k=N k=N 

(2.15) 

Next we look for a uniform estimate with respect to k of E|X_^^-|^p. For s G [—k, —k + 1], 

ArXz'''°dr+ / BrUrdr+Y] / C;A-'^'°dW^;+ V / Dl.UrdW;+ / frdr, (2.16) 

'fe J-k j^j^ J-k J~k J-k 
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where u is the optimal control that minimizes the cost 

-fc+i 



By computing d[{PsX-'''" , X-'''") + 2{f\, X-''^^)] we get, for every T > 

-k+l /--fc+l 



-E J I (l + J2{DiyPsD^)j {B*A+J2iDiy9y)\'ds < 2\E J ^^\rl ^)ds\ < A, 



where A is a constant independent on k. By ()2.16p we get 

/S n — k-\-l n — k-\-l 

sup E\X-''^°\^dr + CE \ur\^dr + E \fr\^dr, 

_ _ -k<r<s J -k J -k 

and so by applying the Gronwall lemma, we get 

-k+l 



sup E\X-'''°\^ <Ce^{A + E f \fr\^dr). 

Cs<-k+l J-k 



-k<s<-k+l J-k 

Since / is stationary, we can conclude that 

sup E\X-'''°f < C, 

-fc<s<-fc+l 

where C is a constant independent on k. By (|2.15p . we get 

M-l 



k=N 

So we can conclude that, fixed i e M, {XZ^'^)n is a Cauchy sequence in L'^{n), and so it converges 
in (O) to a random variable denoted by . Notice that for every t g M we can define Ct , and we 
prove that C'' is a stationary process. Let i S E, —TV < t and s > 0: since the shift 9 is measure 
preserving, 

lim EIX^"^'" 06*^- o 6l,p = 0, 



moreover X^ o tig ^ ^t+s and 



hm E|Xr+™ - CV.P = 0. 

By uniqueness of the limit we conclude that C)' °0s — Ct+s- Notice that since iV e N and JFq ^ ^cT^j 
Co is Jx)-measurable. Let us consider the value of the solution of equation p.l4p starting from 
Xq = Cg. By stationarity of the coefficients and of C,^, we get that X is a stationary solution of 
equation (|2.14p . that we denote by X"^. In order to show the uniqueness of the periodic solution it 
is enough to notice that if = 0, j = 1, 2, and X"^ is a periodic solution of l|2.14p . then 

E|X5|2 = E|X^|2 < Ce~'^^E|C^|2. 

Therefore X^ = and this concludes the proof. □ 

We can now treat the following optimal control problem for a stationary cost functional: minimize 
over all admissible controls u the cost functional 

j\u,X)=E f [\^/s'sXs\^ + \us\^\ds, {u,X)eU\ (2.17) 

where 







W = {iu,X) e Ll,{n X [0,1]) X C{[0,l],L^ifl)) : X^ = Xqo 61,, Vs e M} (2.18) 
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and X is the solution of equation 

d d 

dXt = AtXtdt + Btutdt + ^ ClXtdWl + ^ D\utdWi + ftdt, (2.19) 

i=l i=l 

relative to u. 

Theorem 2.11. Let X^ e C([0, 1], ^^(pl^)) be the unique stationary solution of equation (j2.14p and 
let 

/ + ^ J [PtBt + E (Q* A' + {ClYQtDl) ) + BrrS +Y.^D\rgY. 

1=1 / \ i=l / i=l 

(2.20) 

unique optimal couple for the cost (|2.17p . f/iai is 



J^{u^,X^)^ inf JHu,X). 



T/ie optimal cost is given by 
1^ ^j\u\X^) 



2E [\rlf,)ds-E f \{I+Y.{Diy P,D\)-\Blr\+Y.{Diy gY)\^ds. (2.21) 
Proof. By computing d(PsXs,Xs) + 2(r^,Xs) we get 

E / [{S,X,,Xs) + \us\^]ds = E{PoXo,Xo)-E(PiXi,Xi) + 2E{rlXo)-2E{r\,Xi)-2E f {rlf,)ds 
Jo Jo 



(t,b, + {qIdi + (ciyPsDi) ^ Xs + B:r\ + j2 DUgi'r'^ 

f\(i+t.iDsyPsD')j {B*A+J2{Diy-gy)\'ds. 

■^0 V ^=l J t=l 



'ds 



E 



Since by Propositions [131 HHl and lTTUl fM. X) € U\ we get 

E^ [{SsXs,X,) + \u,\^]ds^~2Ej^ {rlfs)ds + Ej^ \ + Y.^{D\y P ^Dl^ x 
X L. + {I + Y,{D\yP,D\)-^ (p,B,+Y.{Q\Dl + {CiyPsD^y)\ X^+Blr^+j^Dli^yA \^ds 

\ 1=1 \ i=l / i=l / 

-E f\(l + Y.{DiyP,Dl\ {B:r\+Y.{DiygY)?ds. 

■^0 V ^=l J ^=l 

So 

= -(l + Y. (Dl) * PtDl] (ptB, + ^ (qIdI + (CI) * Q.Di)] X\ + B^A +jliD\rgY 



\ i—1 / \ i—1 / i—1 

(2.22) 

is the optimal cost: minimizes the cost p.2ip . and the corresponding state X'' is stationary by 
Proposition mini so that {u\X^) eU^. □ 



3. Ergodic control 

In this section we consider cost functionals depending only on the asymptotic behaviour of the 
state (ergodic control). Throughout this section we assume the following: 

Hypothesis 3.1. The coefficient satisfy hvvothesis \2.^ and moreover 

• S > el, for some e > 0. 

• {A, B, C, D) is stabilizable relatively to S . 

• The first component of the minimal solution P is bounded in time. 

Notice that these conditions implies that {P, Q) stabilize (A, B, C, D) relatively to the identity. 
We first consider discounted cost functional and then we compute a suitable limit of the discounted 
cost. Namely, we consider the discounted cost functional 

r+oo 

MO,x,u)=E e-2-[(5,X°'^■^X°■-'") + |M,|2]ds, (3.1) 
Jo 

where X is solution to equation 

d 

dXs = {AsXs + BsUs)ds + {CiXs + Dlus) dW^, + fsds s > t 
Xt^x. 

A, B, C and D satisfy hvpothesis 12.21 and / e L^(f7 x [0, +oo)) and is a stationary process. When 
the coefficients are deterministic the problem has been extensively studied, see e.g. [L and 

Our purpose is to minimize the discounted cost functional with respect to every admissible control 
u. We define the set of admissible controls as 

W = |w e L^n X [0,+oo)) : E^^ e-2"-[(^^X°'"^", + \u,f]ds < +oo| . 

Fixed a > 0, we define X^ — e^^'^Xg and u" = e~'^^Us. Moreover we set A" = Ag — al and 
ja ^ e-^Vs, and /" e L^^{n x [0, +oo)) n L^(r2 X [0, +oo)). Xf is solution to equation 

d 

dXf = {A'^Xf + BsU'^)ds + Y.i'='l^s+Dlu'^)dWl + fys s>0 ^^^^^ 



1=1 



X^ 



By the definition of X", we note that if {A, B, C, D) is stabilizable with respect to the identity, then 
{A°',B,C,D) also is. We also denote by {P^jQ") the minimal solution of a stationary backward 
Riccati equation (|2.3p with A" in the place of A. Since, for < a < 1, A°' is uniformly bounded 
in a, also is uniformly bounded in a. Arguing as in Proposition 12. 4[ {P",Q") is a stationary 
process. 

Let us denote by {r^^g") the solution of the infinite horizon BSDE 

d ^ d 

drf = -iHn*rfdt - P^^dt - J2 {Kr) * g^dt + Y.dt'^'^W^' ^ ^ (3-3) 

1=1 i=l 

where and K" are defined as in (|2.9p . with A°', P" and Q" respectively in the place of A, P 
and Q. By [3], section 4, we get that equation (|3.3p admits a solution {r",g") £ L'^{Q x [0, +oo)) n 
L^{n X [0,+(X))) X L^{n X [0,T]), for every fixed T > 0. 
Moreover by [4], section 6, we know that 

lim ^ ,pQ inf Ja{0,x,u°') — 

lim„^„[a / 2E{rfJ^)ds-aE / \{I + (DI)* P^ Diy^Btrf (Dl)* gr)\'ds]. 

■^0 ■'O »=1 ^=l 



We can also prove the following convergence result for (r", 5"). 
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Lemma 3.2. For all fixed T > Q, r'^ |[o,t]^ 



[0,T] 



L^{fl X [0,T]). Moreover, for every fixed 



T > 0, as a — > 



/ |(/ + ^ {DlY P^Dir\B:r^ + ^ (DlYgfni'ds 



Jo 



Proof. The first assertion follows from lemma 6.6 in [4J. Notice that stationarity of the coefficients 
in the limit equation gives stationarity of the solution, and so it allows to identify the limit with the 
stationary solution of the dual BSDE. For the second assertion for the optimal couple {X", u") for 
the optimal control problem on the time interval [0, T]: 



[IVKX^I'' + Kpds = (P^x.x) + 2{rS,x) + 2E / {rfj:)ds 

Jo 

E{P^X^,X^)+2E{r^,X^)-E f \{I + J2 {Dl)* P^-Diy^B^r^ +J2 {Dl)* gm'ds. (3.4) 

Since, as a — > 0, in (|3.4p all the terms but the last one converge to the corresponding stationary term, 
and since by [1] (r", g") is uniformly, with respect to a, bounded in Lp(ri x [0, T]) x L'^{Q x [0, T]), 
then (r" |[o.t],5" I[o,t]) ^ {r^ \[o,t],9^ |[o,t]) in L^A^ x [0,r]) x L^ft x [0,T]), we get the desired 
convergence. 



□ 



This is enough to characterize the ergodic limit. Indeed we have that: 
Theorem 3.3. We get the following characterization of the optimal cost: 



lim 2a inf Jaix.u) = E 



2(/(0),rH0)) - |(/ + ^ (i?^)*Poi?S)-^(i3o^& + E (^o)*5S'^)l' 

i=l i=l 

Proof. Let us define rf = e"*r", Ijf = e'^^gf. {rf ,gf) is the solution to 

d ^ d 

drf = -{HtYrfdt + alTfdt - P^ ftdt - ^ (Kr) * g^dt + jy^dWi, t > 0, 



and so, arguing as in lemma (r",g") are stationary processes. Now we compute 

f + OC 



lim„^Q2Q! inf Jq (0, a;, u") =lim^^g 



2a 



= lim 



2a / e-'"^E\{I + J2{Diy P:Dl)-\B:r^ +J2{Diygr)\'ds 

20^6-2"'= / e-^°"2E{7f,fs)ds 
k=i -^0 

OO »\ d d ' 

2a 5^6-2"'= / e-^'^^m PsDi)-\Bt¥: +Y.Wgr)?ds 

k=l •'^ 4=1 i=l 

oo „i 

2aVe-^"'= / 2E(r^,/f)ds 
fe=i -^0 

OO »i d d 

2aY,e~^"^ / E|(/ + E(l?:)*P,"i^:)-i(BX+E(i^:)*5r)Prf^ 

k=l •'^ i=l i=l 



= lim 
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Since (rf^gf) {rl,gl) in Ll,{Q x [0, 1]) x Lj,{n x [0, 1]) we get that 

lim„^o2a mf (0, li") - 2E [\rlf,)ds-E [' \ {I + J2 {Dl)* P:Dir'iB:r^^ +J2 {DlT 9y)\'ds 

i=l i=l 

where the first equahty holds also in the periodic case and the second equality holds only in the 
stationary case. □ 

The next step is to minimize 

J(a;, u) = lim ^^Q2a J(x, u) 

over all u G Z^, where 

Z7 = |u e iL : ^£ e-2"-■[(5,X°■"■^X0^"^") + \us\^]ds < +00, Va > O.j (3.5) 
We will prove that 

inf J{x,u) = J^{u). 

Let X be solution of 

d d d d 

dX- = HsX^ds + Y^KiX^dWl + B,{Btr\ + Y,DlgY)ds + /.ds + + 

i=l -i— 1 2—1 i=l 

and let 

d 

i=l 

Notice that by proposition 12 . 101 if x = then X'-'O is stationary and (u^o, X^o) is the optimal couple 
{ul,xl). 

Lemma 3.4. For all x e L'^{VL), £lA and J{u^ , x) does not depend on x. 
Proof. Let us consider X^'^ the solution of equation 

d 

dXt^'' = HtXt'^'dt + Y,KlXl'"dWi 

i=l 

starting from x at time s. We denote, for every < s < t, C/(i, s)x :— X^'^ . We notice that 

X°'^ - X^'^° = a; - CS + r Hs{X^,'^ - X°'^°)ds + V Kl{X°^- ~ X°'^'")dW: = U{t, 0)(x - Cl)- 

Jo Jo 

So by the Datko theorem, see e.g. 0] and [5j, there exist constants a, C > such that 
So 

E|X="P < Ce"''*E|a;-C5P+]E|X'^op < C, 

where in the last passage we use that X'^ = X^ and it is stationary. 
Again by applying the Datko theorem we obtain 

/•oc 

hmaE / e-^''%2{SXlU{s,0){x-C^o)) + \^U{s,0){x-C^o))\^)ds = 0. 

Moreover 

ut ^ 4 - Ht,Pt,Q,)UiO,t)ix - C^o) 
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It is clear that belongs to the space of admissible control space U. 

The term ut = A(t, Pt, Q()J7(0, t){x — Co); ^ (0 + °°) be proved to be the optimal control 
for the infinite horizon problem with / = and random initial data a: — Co • 



r + oo 

inf E / {\VSsXlf + \uis)\^)ds. 



ueL^((0, + oo);K''-) Jo 

Hence Theorem 5.2 of [4] can be extended without any difficulty to get that: 
J(0, X - Cl n) - E{Po{x -(:l),x- C5) + 2E(ro, x ~ CS> 



E / \{I + J2iDiyPtDl)-\B:n+J2iDiy9l)\'ds. 



Therefore 

/>oo />oo 

E/ e-2"''|u(s)|2rfs < E / |u(s)pds<C. 
Jo Jo 
This proves that u is an admissible control since it follows that 

/"OO 

lim 2aE / e'^^'Hulf - |u^|')rfs = 0. 
Jo 



□ 



We can now conclude as follows: 
Theorem 3.5. For all x G i^(ri) the couple {X^,u^) is optimal that is 

J{u^,x) = min{ J(u, x) : u G U} 

Moreover the optimal cost, that does not depend on the initial state x, is equal to the optimal cost 
for the periodic (respectively stationary) problem, i.e. 

J{u^,x)^l^. 

Proof. We denote irAu^w Ja{x,u) := J*{x,u). If u G U, then for every a > 0, u € U" . Conse- 
quently for every a > 

2aJa{u,x) > 2a J*. 
By taking the limit on both sides we get 

J{x,u) = lim^^g2ajQ > lim^_.o2aJ* = j\ 

By the previous lemma J{x,u^) is independent on a; so we let x = Coi which implies that = u'' 
and = Then 

JiClu^) = hm 2« / e-2"*[|\^t^tV + \4ndt 
"^0 Jo 

= lim 2a(5]e-2'=") / e-''^'[\VS,x'A' + \4\']dt = j\ 



and this concludes the proof. □ 
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